" Software Course on Neural Information Dynamics with Trentool, the Java Information Dynamics Toolkit and Mute "

Speaker: Adam Barett (Department of Informatics, University Sussex, UK) Synergistic and redundant information sharing in Gaussian systems To fully characterize the information that two `source' variables carry about a third `target' variable, one must decompose the total information into redundant, unique and synergistic components. However Shannon's theory of information does not provide formulae to fully determine these quantities. I introduce some recently proposed approaches to addressing this, and apply them to systems of interacting Gaussian variables. Subject to reasonable axioms, I show that for a broad class of Gaussian systems: (i) redundancy reduces to the minimum information provided by either source variable, and is independent of correlation between sources; (ii) synergy can either increase or decrease with correlation between sources. I demonstrate that very simple Gaussian systems can exhibit net synergy, i.e. have the information carried jointly by both sources be greater than the sum of informations carried by each source individually. I discuss implications for measures of information transfer and information-based measures of complexity, both generally and within a neuroscience setting. Importantly, the formulae for separately quantifying synergy and redundancy on continuous time-series data pave the way for new, more detailed approaches to characterizing and quantifying information sharing amongst complex system variables. Speaker: Demian Battaglia (Max Planck Institute for Dynamics and Self-Organization, Nonlinear Dynamics Department Göttingen, Germany) "Liquid" functional interactions across multiple brain scales The structural connectome, one of the "holy grails" of recent large-scale initiatives in neuroscience, plays an important role in shaping functional interactions between neurons or neuronal populations. However the link between structural and (directed) functional connectivity is indirect and mediated by the emergent complexity of the dynamics developed by the considered neuronal circuits. In particular, different dynamical states can give rise to radically different topologies of functional interaction in virtual absence of structural changes. The spontaneous activity of neural circuits has been interpreted as a noise-driven exploration of a landscape of internally-available meta-stable dynamical states, which are transiently visited over time in a stochastic fashion. If a correspondence exists between dynamical states and functional interaction modes, we would thus expect a marked and characteristic time-dependency of functional connectivity networks, possibly manifesting switching behavior corresponding to inter-state transitions. We explore functional connectivity dynamics in systems at different spatial-scales, analyzing: at the micro-scale of the local circuit, LFP and spike train recordings from enthorinal cortex in anaesthesized rodents; at the meso-scale of cortical networks, multi-channel LFP recordings from fronto-parietal networks of macaque monkey during a working memory task; and, finally, at the whole-brain scale, resting state human BOLD imaging data. At all these scales of analysis, we find that functional connectivity is highly dynamic and presents signatures of switch-like transitions. Focusing on the macro-scale, we show that resting state functional connectivity switching can be accounted for by mean-field computational models of large-scale cortical networks, provided that the repertoire of possible dynamical states is maximized by operating in a subcritical regime, rather than a strictly critical one (at contrast with previous studies, focusing on the rendering of static aspects of resting state). Finally, we stress how functional connectivity dynamics correlates with subjects' age, providing a novel source of biomarkers and additional information on the aging process complementary to what revealed by changes of structural connectivity or of time-averaged functional connectivity. Speaker: Luca Faes (BIOtech – University of Trento, and IRCS Program PAT-FBK, Trento, Italy) Detecting the Causal Structure of Physiological Networks through Internal Information and Information Transfer In the emerging field of information dynamics, the analysis of coupled dynamical systems can be performed in a natural way by decomposing the predictive information about an assigned target system into amounts quantifying the information stored inside the system and the information transferred to it from the other systems. While the information transfer can be associated (though with some caution) to the presence of causal interactions between the observed systems, the information storage incorporates dynamical influences arising not only from the investigated target system, but also from the other systems potentially connected to it. As an alternative to the information storage, a measure of ‘internal information’ may be defined in a way such that it reflects more closely the internal dynamics of the target system, that is the dynamics arising from interactions between the present and the past of the dynamic process descriptive of the system states. In this context, my talk will be concerned with the evaluation of internal information and information transfer as measures related to the causal statistical structure of coupled dynamic processes. After defining operative measures of internal information and information transfer (namely, conditional self entropy and transfer entropy), the properties of these measures as quantities reflecting internal dynamics and causal interactions will be investigated for benchmark systems allowing exact theoretical computation. Then, practical computation will be considered proposing two estimation approaches: a model-based approach working under the linear Gaussian assumption and a model-free approach based on nearest neighbor entropy estimation and non-uniform embedding. The two approaches will be exploited for the quantitative analysis of brain-heart and brain-brain interactions during sleep. The analysis will be performed on the time series of the cardiac autonomic activity and of the EEG brain activity assessed in the delta, theta, alpha, sigma and beta bands, measured from the polysomnographic recordings of ten healthy subjects. The comparison between model-based and model-free estimates of internal information and information transfer will lead to highlight the role of nonlinear dynamics in the functional networks describing the joint modulation of cardiac and brain functions during sleep. Speaker: Esther Florin (Department of Neurology, University Hospital, Cologne, Germany) Guidelines for the use of multivariate Granger causality Several multivariate Granger causality-based connectivity measures have been suggested to assess directionality in electrophysiological data. Two of the most used ones are the directed transfer function and partial directed coherence. Still the applicability and limitations of Granger based analysis in different contexts are not always clear. In my talk, I will present results on the performance of different Granger causality-based measures for neural data as well as the influence of common pre-processing techniques in electrophysiological data. In particular, the dependence upon data length, noise level, coupling strength, and model order will be discussed. Lastly the use of filters and their influence on Granger causality will be presented. Based on these previous results, I will present an application of causality measures to intraoperatively obtained local field potentials from Parkinson’s disease patients. Speaker: Moritz Grosse-Wentrup (Department Empirical Inference, Max Planck Institute for Intelligent Systems, Tübingen, Germany) Causal interpretation of encoding and decoding models in neuroimaging Even though neuroimaging models per se only provide insights into the neural correlates of cognition, causal terminology is often introduced in the interpretation of neuroimaging data. In this talk, I investigate which causal statements are warranted and which ones are not supported by empirical evidence. I argue that in order to decide this question it is necessary to distinguish between encodingand decoding models as well as between stimulusand response-based experimental paradigms. I then proceed to demonstrate that only encoding models in stimulus-based paradigms support unambiguous causal statements. By combining encoding and decoding models trained on the same data, however, it is possible to obtain insights into causal structure beyond those implied by each individual model type. I illustrate the empirical significance of our theoretical findings on EEG data recorded during a visuomotor learning task. Speaker: Dimitris Kugiumtzis (Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki, Greece) Direct information flow in high-dimensional time series and scalp EEG A currently important issue with high relevance to many applications, such as neuroscience and finance, is the estimation of direct information flow in multi-variate time series of many variables. Direct information flow is the information about the evolution of one variable or subsystem provided by another variable or subsystem that is not shared by the rest observed variables. When many variables are simultaneously observed at fixed time intervals, such as the multi-channel electroencephalogram (EEG), the estimation of the direct information flow is a challenging task and involves estimating issues, namely the selection of free parameters depending on the method of choice, the curse of dimensionality, and the bias of assigning indirect effects and external source effects to direct effects. These issues unavoidably give rise to inaccurate formation of complex networks (nodes are the variables and connections are determined by the information flow measure) that fail to preserve the original coupling structure of the system of the observed variables. I will discuss these problems, assess the current methodology and focus particularly on the direct information flow measure of partial mutual information from mixed embedding (PMIME), recently proposed by our group. I will then present some new results about the ability of PMIME and other methods in identifying correctly the coupling structure of an observed high-dimensional dynamical system. Further, I will explore the shortcomings of the current methodology in estimating effective connectivity from multi-channel scalp EEG. Speaker: Andreea Lazar (Max Planck Institute for Brain Research, Frankfurt, Germany) Computation at the Edge of Chaos in Recurrent Neural Networks Recurrent neural networks (RNNs) are powerful computational tools that can exploit long-term dependencies in the input when calculating the output. RNNs perform well on complex temporal tasks, for which feed-forward architectures are ill-equipped. However, training or optimizing such models is challenging and often very slow. It has been proposed that neural networks whose dynamics are situated at the edge of chaos demonstrate improved computational capabilities. Interestingly, cortical networks have been shown to exhibit similar critical dynamics. In contrast to most of their artificial counterparts, biological networks are continuously refined by various forms of plasticity that act in a reproducible, highly specific fashion. We explore the interplay between structure, dynamics and computational power in both random RNNs and brain-inspired self-organizing recurrent networks (SORNs) shaped by plasticity. We find that general purpose machines are best tuned when they exhibit dynamics at the edge of chaos. However, networks that learn the statistical structure of their inputs and internalize a model of the environment, outperform non-plastic networks and develop subcritical dynamics. Speaker: Anna Levina (Max Planck Institute for Dynamics and Self-Organization, Nonlinear Dynamics Group, Göttingen, Germany) Dynamical range in critical and near-critical systems Self-organized criticality is a common phenomenon in nature and became a fascinating research subject in the field of neuroscience, when critical avalanches were predicted theoretically and observed experimentally to occur in networks of neurons. It was expected, that criticality will bring neuronal system into computationally optimal state, but so far the only clear cut optimality associated with critical avalanches is the optimality of improved dynamical range. Here I present numerical results and analytical computation of dynamic range for different types of models used to study avalanches. Surprisingly, in some models dynamical range and closeness of avalanches distribution to a power law (that is often used to define criticality) are optimized for different parameter values. I discuss conditions for the coincidence of this two measures. Speaker: Daniele Marinazzo (Department of Data Analysis, Faculty of Psychological and Pedagogical Sciences, University Gent, Belgium) Synergy and redundancy in the Granger causal analysis of dynamical networks We analyze, by means of Granger causality (GC), the effect of synergy and redundancy in the inference (from time series data) of the information flow between subsystems of a complex network. While we show that fully conditioned GC (CGC) is not affected by synergy, the pairwise analysis fails to prove synergetic effects. In cases when the number of samples is low, thus making the fully conditioned approach unfeasible, we show that partially conditioned GC (PCGC) is an effective approach if the set of conditioning variables is properly chosen. Here we consider two different strategies (based either on informational content for the candidate driver or on selecting the variables with highest pairwise influences) for PCGC and show that, depending on the data structure, either one or the other might be equally valid. On the other hand, we observe that fully conditioned approaches do not work well in the presence of redundancy, thus suggesting the strategy of separating the pairwise links in two subsets: those corresponding to indirect connections of the CGC (which should thus be excluded) and links that can be ascribed to redundancy effects and, together with the results from the fully connected approach, provide a better description of the causality pattern in the presence of redundancy. Applications to real biological datasets are also presented. Speaker: Oliver Obst (CSIRO ICT Centre, Australia) Representations and Information Dynamics in Recurrent Networks Various forms of plasticity are hypothesised to play a crucial part for information processing in the brain, e.g., for information storage and information transfer between different areas. Synergetic effects between different plasticity mechanisms have also shown to be responsible for emergence of distinct representations for an object dependent on its context in the input stream of a recurrent network. We look at the effects of plasticity mechanisms with respect to the information dynamics in the system, and are able to quantify the contribution of plasticity mechanisms to changes in information storage and information transfer. Speaker: Gordon Pipa (Institut für Kognitionswissenschaft, Universität Osnabrück, Germany) D2IF A statistical framework to infer delay and direction of information flow from measurements of complex systems Neuronal activity is measured as from spatially distributed subsystems with complex interactions, weakly coupled to a high-dimensional global system. We present here a statistical modeling framework, D2IF (Delay & Direction of Information Flow), to estimate the direction of information flow and its delay in measurements from systems of this type. Informed by di_erential topology, Gaussian process regression is employed to reconstruct measurements of putative driving systems from measurements of the driven systems. These reconstructions serve to estimate the delay of the interaction by means of an analytical criterion developed for this purpose. The model accounts for a range of possible sources of uncertainty, including temporally evolving intrinsic noise, while assuming complex nonlinear dependencies. Furthermore, we show that if information flow is delayed, this approach also allows for inference in strong coupling scenarios of systems exhibiting synchronization phenomena. The general validity of the method is established on a variety of delay-coupled chaotic oscillators. In addition, we show that these results seamlessly transfer to real data generated from local field potentials in cat visual areas. Speaker: Alberto Porta (Department of Biomedical Sciences for Health, University of Milan, Milan, Italy) Non Uniform K-Nearest-Neighbors Causality Approach for the Assessment of Neural Sympathetic Control of Circulation during Orthostatic Challenge During orthostatic challenge venous return is reduced, as a result of blood pooling in the legs, and cardiac output tends to decrease, thus resulting in a possible arterial blood pressure drop and consequent orthostatic syncope. To counteract this tendency sympathetic control is activated leading to an increased heart rate and peripheral vasoconstriction in the attempt to favor venous return, maintain cardiac output and, consequently, arterial blood pressure levels. Nonlinear causality analysis provide an ideal framework to assess the strength of the physiological interactions aiming at the maintenance of arterial blood pressure during orthostatic challenge in presence of relevant nonlinear relations and confounding factors blurring the causal link from sympathetic neural activity to arterial blood pressure (e.g. the direct effect of heart rate and respiration on arterial blood pressure). We will exploit a previously proposed Granger causality approach based on a k-nearest neighbors approach (see A. Porta et al, PLoS ONE, 9, e89463, 2014) to estimate the importance of the causal relation from sympathetic activity to arterial blood pressure in a multivariate framework via the computation of local non linear prediction and transfer entropy. In a group of healthy humans the strength of the relation from sympathetic activity, as measured by placing a tungsten electrode in a fascicle of the right peroneal nerve, to arterial blood pressure will be computed by accounting for the influence of heart rate and respiration during head-up tilt test. Tilt table inclination will be varied to modulate the importance of the orthostatic challenge. The two paradigms for causality analysis (i.e. the local nonlinear prediction and transfer entropy) will be compared in relation to their power in describing the neural sympathetic control of circulation during the considered cardiovascular challenge. Speaker: Viola Priesemann (Max Planck Institute for Dynamics and Self-Organization, Nonlinear Dynamics Group, Göttingen, Germany) Collective neural dynamics and information processing The human brain has amazing information processing capabilities, which rely on the activity of 80 billion neurons, each of them interacting with thousands of other neurons. However, their collective dynamics and the origin of their information processing capabilities remain unknown. A popular hypothesis is that the collective dynamics reflects a critical state, because in models this state showed maximal processing capacity (quantified as information storage and transfer). However, I recently provided evidence that in vivo spike recordings from rats, cats and monkeys reflect a subcritical regime instead of criticality. Moreover, the ‘distance to the critical state’ changed from wakefulness to deep sleep. This suggests that the brain may change its distance to the critical state depending on needs: In general, it may maintain a safety margin to criticality, because criticality comes with the risk of runaway activity (epilepsy), but it reduces this safety margin temporarily when high processing capacities are needed. Speaker: Karin Schiecke (Institute of Medical Statistics, Computer Sciences and Documentation, Jena University, Germany) Convergent Cross Mapping: basic concept and practical application Convergent Cross Mapping (CCM), introduced by Sugihara et al. [1] in 2012, tests for causation between time series X and Y by looking at the correspondence between so-called “shadow manifolds” M x und M y constructed from lagged coordinates (nonlinear state space reconstruction) of the time series variables of X and Y . Running contrary to intuition (and Granger causality) the basic concept of CCM is that when causation is unilateral ( X drives Y), then it is possible to estimate X from Y , but not Y from X . Therefore, CCM is measuring the extent to which the historical record of Y values can estimate the states of X (cross mapping of X by using M y : X /M y ) or vice versa (cross mapping of Y by using M x : Y /M x ). Practically, correlation coefficient between original and estimated time series ( X and X /M y or Y and Y /M x respectively) are examined using increasing data length („library length“). In bidirectional case ( X drives Y stronger than vice versa) correlation between X and X /M y should converge faster / reach a higher plateau than correlation between Y and Y /M x respectively. The performance of CCM depends on estimation parameters like embedding dimension, time lag, library length as well as used error metrics. Furthermore, system noise is strongly influencing the estimation of CCM. Aim of our study is to demonstrate potentials and limitations of CCM. Simulated data are generated on base of a coupled two-species non-linear logistic difference system with chaotic dynamics. Varying parameters of state space reconstruction as well as the influence of different levels of additive noise are investigated. Results of CCM are compared to the one of Granger causality and Transfer Entropy investigation. Additionally, a simple surrogate data test using phase randomization is introduced for statistical evaluation of convergence of correlation coefficients with increasing library length. Analyses of CCM applied to real data will follow. [1] Sugihara G, May R, Ye H, Hsieh CH, Deyle E, Fogarty M, et al. Detecting Causality in Complex Ecosystems. Science. 2012;338(6106):496-500. Speaker: Sebastiano Stramaglia (Department of Physics, University of Bari Aldo Moro, Italy) Structural constraints to functional interactions in the brain (at the macroscale) The topology of the underlying structural brain network shapes the pattern of functional interactions among brain areas. Two related examples will be described, dealing with (i) information transfer and criticality in the Ising model on the Human Connectome, and (ii) the comparison between brain structure and function at the aggregate level. Speaker: Gasper Tkacik (Biophysics and Neuroscience, Institute of Science and Technology Austria ) Critical behavior in networks of real neurons The patterns of joint activity in a population of retinal ganglion cells encode the complete information about the visual world, and thus place limits on what could be learned about the environment by the brain. We analyze the recorded simultaneous activity of more than a hundred such neurons from an interacting population responding to naturalistic stimuli, at the single spike level, by constructing accurate maximum entropy models for the distribution of network activity states. This -essentially an "inverse spin glass" -construction reveals strong frustration in the pairwise couplings between the neurons that results in a rugged energy landscape with many local extrema; strong collective interactions in subgroups of neurons despite weak individual pairwise correlations; and a joint distribution of activity that has an extremely wide dynamic range characterized by a zipf-like power law, strong deviations from "typicality", and a number of signatures of critical behavior. We hypothesize that this tuning to a critical operating point might be a dynamic property of the system and suggest experiments to test this hypothesis.